Heuristically Accelerated Reinforcement Learning: Theoretical and Experimental Results
نویسندگان
چکیده
Since finding control policies using Reinforcement Learning (RL) can be very time consuming, in recent years several authors have investigated how to speed up RL algorithms by making improved action selections based on heuristics. In this work we present new theoretical results – convergence and a superior limit for value estimation errors – for the class that encompasses all heuristicsbased algorithms, called Heuristically Accelerated Reinforcement Learning. We also expand this new class by proposing three new algorithms, the Heuristically Accelerated Q(λ), SARSA(λ) and TD(λ), the first algorithms that uses both heuristics and eligibility traces. Empirical evaluations were conducted in traditional control problems and results show that using heuristics significantly enhances the performance of the learning process.
منابع مشابه
Heuristically Accelerated Q-Learning: A New Approach to Speed Up Reinforcement Learning
This work presents a new algorithm, called Heuristically Accelerated Q–Learning (HAQL), that allows the use of heuristics to speed up the well-known Reinforcement Learning algorithm Q–learning. A heuristic functionH that influences the choice of the actions characterizes the HAQL algorithm. The heuristic function is strongly associated with the policy: it indicates that an action must be taken ...
متن کاملThe Use of Cases as Heuristics to speed up Multiagent Reinforcement Learning
This work presents a new approach that allows the use of cases in a case base as heuristics to speed up Multiagent Reinforcement Learning algorithms, combining Case Based Reasoning (CBR) and Multiagent Reinforcement Learning (MRL) techniques. This approach, called Case Based Heuristically Accelerated Multiagent Reinforcement Learning (CB-HAMRL), builds upon an emerging technique, Heuristic Acce...
متن کاملCase-Based Multiagent Reinforcement Learning: Cases as Heuristics for Selection of Actions
This work presents a new approach that allows the use of cases in a case base as heuristics to speed up Multiagent Reinforcement Learning algorithms, combining Case-Based Reasoning (CBR) and Multiagent Reinforcement Learning (MRL) techniques. This approach, called Case-Based Heuristically Accelerated Multiagent Reinforcement Learning (CB-HAMRL), builds upon an emerging technique, Heuristic Acce...
متن کاملHeuristic Q-Learning Soccer Players: A New Reinforcement Learning Approach to RoboCup Simulation
This paper describes the design and implementation of a 4 player RoboCup Simulation 2D team, which was build by adding Heuristic Accelerated Reinforcement Learning capabilities to basic players of the well-known UvA Trilearn team. The implemented agents learn by using a recently proposed Heuristic Reinforcement Learning algorithm, the Heuristically Accelerated Q–Learning (HAQL), which allows th...
متن کاملHeuristically-Accelerated Reinforcement Learning: A Comparative Analysis of Performance
This paper presents a comparative analysis of three Reinforcement Learning algorithms (Q-learning, Q(λ)-learning and QSlearning) and their heuristically-accelerated variants (HAQL, HAQ(λ) and HAQS) where heuristics bias action selection, thus speeding up the learning. The experiments were performed in a simulated robot soccer environment which reproduces the conditions of a real competition lea...
متن کامل